Model Sensitivity in Panel Data Analysis: Some Caveats About the Interpretation of Fixed Effects and Differences Estimators

This paper shows that first-differences or fixed-effects models may understate the effect of interest because of the variation used to identify the model. For example, much of the recent research estimating the relationship between AFDC benefits and fertility has used fixed-effects models. If the cross-sectional variation in AFDC benefits is removed, then the variation over time within states left for identification will largely reflect transitory, idiosyncratic changes in welfare benefits, which are unlikely to affect fertility. In some sense, the independent variable as being measured with error, since it is contaminated with these temporary fluctuations. Estimates obtained using fixed-effects models are small because the “signal” of permanent changes is being overwhelmed by the “noise” of transitory changes. Two empirical examples are presented: one on the relationship between AFDC and fertility and the other on the relationship between local economic conditions and AFDC participation. Coefficient estimates from first-differences, long-differences, and fixed-effects models are compared. These estimates differ in ways that are consistent with the presence of measurement error. Attempts are made to obtain consistent estimates using instrument variables. Because no valid external instrument is apparent in the first example, the lagged internal instruments suggested by Griliches and Hausman (1986) are considered, but the analysis suggests that most of these instruments are “weak.” In the second example, the decline of the steel industry is used as an instrument that is correlated with the permanent change in the prospects of low-skill workers. Estimates from the 2SLS model are substantially larger in magnitude than those obtained with first-differences or fixed-effects.